Designing for Augmented Cognition - Problem Solving for Complex Environments

The objective of this paper was to aggregate research done during several different Small Business Innovative Research (SBIR) grants as they apply to the design of complex environments and Augmented Cognition. This paper provides a high level exploration of the definition of situational awareness (SA), the action loop, an advanced mitigation framework, and a repeatable design methodology that was created to overcome several key mistakes made by UI designers. The discussion is illustrated using a recent User Interface Metaphor design project that maximizes information flow in a novel F-35 Joint Strike Fighter Cockpit. While testing is not complete on the resulting UI metaphor, initial observations indicate that the results of using these models and processes offer a significant improvement in performance and user acceptance is appears to be high. 1. THE FUTURE OF THE HUMAN-COMPUTER DYAD The User Interface designer’s job is becoming more difficult. Moore’s law states the processing capabilities of computers will double every 18 months [1]. This has proven true for the last 40 years. It follows that as computers become more powerful the amount of information available in real-time will also increase. As complex domains such as the fighter cockpit continually increase the amount of data available to the user the inherent computer-to-human bottleneck will be aggravated at the expense of user efficiency. To this day computer applications have depended overwhelmingly on visual, verbal information, and have relied primarily on serial communication [5]. A properly designed complex application will overcome these limitations. Additionally, the downward pressure of ever-present budget cutbacks is making it essential to do more with fewer staff. Augmented Cognition (AugCog) seeks to revolutionize the way humans interact with computers by leveraging human physiological indicators to direct human-systems interaction [8]. This paper seeks to aid the UI designer when confronting an AugCog application. The human-computer dyad provides an interesting model. The human participant has the innate ability to assess complex situations, understand the context, and make appropriate decisions based on his or her aggregate situational awareness. Albers states people viewing information have a goal of locating the relevant information, 2 Joseph Juhnke, Timothy Mills, and Jennifer Hoppenrath mentally forming the relationships within the information, relating it to their realworld situations, and, most importantly, using it to perform a useful/correct action [1]. Unlike humans, the computer participant is particularly adept at processing large amounts of tabular data and summarizing it in more digestible forms. Neither participant shares the other’s strengths. Together this complementary relationship has potential for significant increases in performance. As Benyon states, to accomplish this synergy “we must shift attention from humans, computers and tasks to communication, control and the distribution of domain knowledge [2]. 2. DEFINING INTERACTION MODELS There are three models that must be considered when designing the user interface metaphor for the human-computer dyad: situational awareness, the action loop, and a mitigation framework. 2.1 SITUATIONAL AWARENESS Wickens defines situational awareness as “the continuous extraction of information about a dynamic system or environment, the integration of this information with previously acquired knowledge to form a coherent mental picture, and the use of that picture in directing further perception of, anticipation of, and attention to current events” [11]. The Air Force Research Laboratory similarly, and perhaps more simply, defines situational awareness as “how accurately a person perceives his current environment relative to the reality of that environment” [9]. Applying situational awareness to the goal of improving user interface, Davenport identifies three key areas of awareness encountered by the human participant: systems awareness, task awareness, and spatial awareness [3]. − Systems Awareness – This is the human participant’s ability understand the state of his or her equipment. In the cockpit, for example, systems awareness is often abstract and usually requires aggregation of various gauge indications. Knowing that the engine is running hot means nothing by itself, but combined with other systems indicators, may indicate a potential problem. − Task Awareness – This is the human participant’s ability to accurately obtain information relating to tasks relevant to his or her goals. Understanding the current state of all tasks that are underway is critical as poor task awareness increases cognitive load, diminishing overall situational awareness. Good task awareness also enables the human participant to make informed decisions when making changes to the planned task. − Spatial Awareness – Spatial awareness can be broken into two sub-categories; Global and Local. Global spatial awareness is an understanding of the position of Designing for Augmented Cognition Problem Solving for Complex Environments 3 the human participant and his or her equipment in the world at that moment. It is the ability to accurately determine relative relationship and trajectory of objects within a global 360-degree sphere of influence and often pertains to the human participant’s relation to a target destination, anticipation of upcoming objects, and other spatial directional judgments. Local spatial awareness pertains to the attitude (vector and velocity) of the human participant’s equipment. This is particularly important when dealing with moving platforms such as aircraft. During observations of pilots using simulator software for the Information Flow SBIR, it was repeatedly noted that during increased times of cognitive load the first errors made were related to spatial awareness. As the local spatial orientation of aircraft changes rapidly and frequently, the related local spatial SA tended to be the first awareness lost. While further empirical evaluation is required, it quickly became apparent that designing an interface metaphor and mitigation plan that works to solve this tendency would be a high priority.